Contrastive Learning with the Feature Reconstruction Amplifier

Contrastive learning has emerged as one of the most promising self-supervised methods. It can efficiently learn transferable representations samples through instance-level discrimination task. In general, performance contrastive method be further improved by projecting high-dimensional into low-dimensional feature space. This is because model more abstract discriminative information. However, when features cannot provide sufficient information to (e.g., are very similar each other), existing will limited a great extent. Therefore, in this paper, we propose general module called Feature Reconstruction Amplifier (FRA) for adding additional model. Specifically, FRA reconstructs embeddings with Gaussian noise vectors and projects them reconstruction space, add designed loss. We have verified effectiveness itself exhaustive ablation experiments. addition, perform linear evaluation transfer on five common visual datasets, experimental results demonstrate that our superior recent advanced